High level test bench generation using software engineering concepts

The tool to be presented in the panel deals with validation of VHDL descriptions at the early phase of the design of a digital system. Our approach consists in generating test data from a given VHDL behavioral description. The validation is achieved by comparing the results obtained using the simulation of the VHDL description within the test data and the results which should have been obtained from the specification of the system to be designed. Our approach is based on software testing concepts. We choose such an approach because a VHDL description is a software program describing the behavior of a digital system. The problem is to generate test data using software engineering concepts. Generating test data points out the resolution of three basic problems : (i) it is necessary to define the number of test data to be considered (this number is called the length of the test data in the following) ; (ii) it is necessary to define criteria which express the “quality” requirements that the test data have to fulfill ; (iii) it is necessary to define an algorithm allowing to generate test data. To solve these problems we are concerned with testing techniques developed in the field of software engineering. This interest is motivated by the fact that behavioral hardware languages such as VHDL and conventional languages such as C or ADA are supported by common concepts. Having selected criteria from the field of software testing allowing the three aforementioned problems to be solved, we are in the phase of studying how such criteria could be measured and applied to VHDL behavioral descriptions. In order to find criteria which could estimate the length of test data and express the quality of test data, we have been concerned with two kinds of techniques: (i) the computation of cyclomatic complexity metric (McCabe metrics) and (ii) the application of coverage-based metrics. The generation of test data is based on a powerful algorithm issued from software testing methods : this algorithm allows to calculate the minimum number of control flow paths which are sufficient for the generation of all possible execution paths involved in a VHDL description. The McCabe metric and the previous algorithm are based on a graphical representation of the control part of the software being tested. McCabe defined a cyclomatic number of a graph associated with the control part of software. This number represents the number of linearly independent paths of the graph. He proved that the cyclomatic number represents the minimum number of test data to be generated in order to test the control part of software. In order to evaluate the quality of test data, conventional software testing criteria are used. These criteria correspond to coverage based metrics. Using the concepts issued from software engineering, it is obvious that theses concepts are easily used on the control flow representation of VHDL descriptions. The only efficient way to verify that a design description written in VHDL operates as expected is to simulate it. To simulate a circuit, the tester will need to develop an additional VHDL program called test bench. We propose in the figure of at side a framework for deriving test benches for VHDL descriptions. The test data generator tool is represented by the grayed box. We are in the phase of validating the software allowing to automatically generate test data for VHDL descriptions from the previous concepts. The software has been realized in common LISP on a SPARC SUN workstation. Comparator